delay)delay over battery - without outliers)This document analyses the self-developed scheduler component in Android-based mobile devices as part of the SyMptOMS project.
To create the PDF of the computational notebook you can run the following commands in a new R session. If you have problems rendering the PDF you can execute each chunk independently in RStudio.
This document does not install the required R packages by default. You can run the script install.R to install all required dependencies on a new R installation, or use install.packages(..) to install missing R packages.
The plots and tables use the packages ggplot2, knitr::kable() and kableExtra.
Required libraries and runtime environment description are as follows.
library(tidyverse)
library(kableExtra)
library(here)
# library(googledrive)
library(lubridate)
library(stringr)
library(scales)
# library(plotly)
library(grid)
library(gridBase)
library(gridExtra)
library(egg)
Log files are stored in shared gdrvie folder. This notebook downloads them locally in the folderdata-raw.
| experiment | gsheets_name | gsheets_link |
|---|---|---|
| #1 | AD_BQ.csv | https://drive.google.com/open?id=1fP9r0S8ORa689yHRtfCtYQothLuGcCw3 |
| #1 | AD_NV.csv | https://drive.google.com/open?id=1l-vnaT-Smy0SegArIz5A108-IQ2_jmVV |
| #1 | AD_A1.csv | https://drive.google.com/open?id=1gzk1ezN5t5yBv2RnbznUqvalDx97eu3o |
| #1 | BA_H9.csv | https://drive.google.com/open?id=1BLXO4Rvz6ppWFJe5oN9Y0hKh__ypofxk |
| #1 | BA_MO.csv | https://drive.google.com/open?id=1RizGjKR8QLOdJna7qGPP7dYX9WWAysO4 |
| #2 | BA_BQ.csv | https://drive.google.com/open?id=1rfaooLZ0Up0gNx1SCA92TV1JFxcLU0sk |
| #2 | BA_NV.csv | https://drive.google.com/open?id=1_KIXX5FGXv7MO37QbUDnd_KGqiGihiMI |
| #2 | BA_A1.csv | https://drive.google.com/open?id=1LezrtyB9i4K7cQsmI78E0v_KaOyhZcXI |
| #2 | AD_H9.csv | https://drive.google.com/open?id=1ISjsfSLZt6n9miryQtDr-tsZV9bfbCDT |
| #2 | AD_MO.csv | https://drive.google.com/open?id=1bx2RVNz05qVKGr-32GI-ZIOKQWlqKBPz |
Raw data files are cleaned and merged together into a single file stored in the folder data. New fields such as dates breakdown are computed too.
Key Variables:
quantitative (interval)
plan_date (datetime): task planning time.exec_date (datetime): task execution time.quantitative (ratio)
delay (numeric): execution delay in seconds ((exec_date - plan_date) - 60).categorital (ordinal)
battery(numeric): battery levelcategorical (nominal)
device_id/device_name: device identifier/nameexp_id: experiment identifier.scheduler: scheduler type.| # exp | Type scheduler | Device | # records | Start | End | Min | Bottom whisker | 1st Qu. | Median | Mean | 3rd Qu. | Top whisker | Max |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| #1 | Advanced | BQ Aquaris V | 21391 | 2020-03-30 12:15:03 | 2020-04-14 13:07:57 | -48.591 | 0.235 | 0.546 | 0.651 | 0.769 | 0.754 | 1.065 | 61.204 |
| Nvidia Shield Tablet | 21453 | 2020-03-30 12:15:03 | 2020-04-14 13:08:14 | -52.811 | -0.882 | 0.052 | 0.241 | 0.673 | 1.043 | 2.518 | 103.849 | ||
| Xiaomi Mi A1 | 21465 | 2020-03-30 12:23:27 | 2020-04-14 13:07:50 | -60.925 | -0.840 | 0.088 | 0.269 | 0.493 | 0.838 | 1.945 | 68.872 | ||
| Basic | Honor 9 | 21574 | 2020-03-30 12:00:03 | 2020-04-14 13:01:35 | -0.824 | -0.094 | 0.068 | 0.087 | 0.311 | 0.187 | 0.364 | 295.963 | |
| Motorola Moto G | 21619 | 2020-03-30 12:00:07 | 2020-04-14 13:01:16 | -0.183 | -0.183 | 0.062 | 0.100 | 0.875 | 0.289 | 0.629 | 60.896 | ||
| #2 | Advanced | Honor 9 | 19377 | 2020-04-15 11:00:02 | 2020-04-29 12:59:15 | -59.535 | -0.423 | 0.167 | 0.225 | 0.591 | 0.842 | 1.853 | 93.032 |
| Motorola Moto G | 19735 | 2020-04-16 12:09:53 | 2020-04-30 12:58:44 | -30.166 | -0.212 | 0.117 | 0.167 | 2.092 | 0.782 | 1.778 | 126.487 | ||
| Basic | BQ Aquaris V | 20146 | 2020-04-15 11:16:26 | 2020-04-29 13:00:20 | -0.111 | 0.065 | 0.280 | 0.297 | 0.390 | 0.424 | 0.639 | 34.412 | |
| Nvidia Shield Tablet | 3528 | 2020-04-15 11:16:18 | 2020-04-17 22:27:06 | -0.871 | -0.871 | 0.021 | 0.122 | 0.431 | 1.007 | 2.085 | 4.841 | ||
| Xiaomi Mi A1 | 10832 | 2020-04-15 11:16:23 | 2020-04-23 00:52:00 | -0.905 | -0.661 | 0.066 | 0.302 | 0.405 | 0.614 | 1.416 | 108.035 |
Observations:
181.120 records in total.
50.6% (91.582/181.120) is the interquartile range or IQR, i.e. the middle 50% of the data.
39.6% (71.769/181.120) represent the reasonable extremes of the data, i.e. these whisker values are the minimum and maximum values that do not exceed 1.5 × IQR.
9.8% (17.769/181.120) are outliers, i.e. values beyond the top and bottom whiskers.
Nvidia Shield Tablet has stopped triggering alarms on 2020-04-17 22:27:06 when running the basic scheduler. Since there’s no watchdog in the basic version, this device is not going to trigger alarms anymore. Died!
Xiaomi Mi A1 has stopped triggering alarms on 2020-04-23 00:52:00 when running the basic scheduler. An explanation may be that Huawei/Honor smartphones on Android 9+ have a Launch Records where the OS records how much times an app has been launched by itself, so the OS has the possibility to detect this kind of apps. In these smartphones, Huawei also introduced a new task killer (Power Genius) which kills everything that is not whitelisted by Huawei, and does not give users any configuration options. Therefore, the Power Genius uses the information from the Launch Records to kill the scheduled alarms
In mental health intervetions, to have a predictable execution of planned actions is preferred since these notifications and/or alerts are delivered at the right moment. It is inconceivable, for example, that a patient gets a notification (action) 1 minute later than the event (she walks past a gambling hall) that triggered it, because the patient could be dozens of meters away from the place of interest or, even worse, walked into that place. Therefore, closing the gap between the triggered event and the action is critical (time matters), especially because time and space/place must be in synchrony. If these tww diemsion are not in synchrony, the intervetion becomes less effective.
Given that average response time (delay) is similar in all devices (< 1second), outliers are a good candidate to study reliability. Fewer outliers would mean responses (delay) more predictable. Therefore, high values and/or frequent outliers would signal lack of synchrony between space/place and time dimensions in an intervention.
| Device | # exp | Type scheduler | # outliers | # outliers positive | # outliers negative |
|---|---|---|---|---|---|
| BQ Aquaris V | #1 | Advanced | 1646 | 925 | 721 |
| BQ Aquaris V | #2 | Basic | 2996 | 2499 | 497 |
| Honor 9 | #2 | Advanced | 307 | 209 | 98 |
| Honor 9 | #1 | Basic | 5033 | 5018 | 15 |
| Motorola Moto G | #2 | Advanced | 2861 | 2852 | 9 |
| Motorola Moto G | #1 | Basic | 4377 | 4377 | NA |
| Nvidia Shield Tablet | #1 | Advanced | 294 | 285 | 9 |
| Nvidia Shield Tablet | #2 | Basic | 3 | 3 | NA |
| Xiaomi Mi A1 | #1 | Advanced | 224 | 121 | 103 |
| Xiaomi Mi A1 | #2 | Basic | 28 | 27 | 1 |
Observations:
Advanced scheduler beats the basic one in terms of number of outliers. Honor 9 shows a remarkly difference.
Majority of outliers are positive (beyond top whisker).
Number of outliers: 4642
Number of outliers (top range): 3424 - 73.8% (3.424/4.642).
Number of outliers (bottom range): 1218 - 26.2% (1.218/4.642).
Number of outliers: 297
Number of outliers (top range): 288 - 97% (288/297).
Number of outliers (bottom range): 9 - 3% (9/297).
Number of outliers: 252
Number of outliers (top range): 148 - 58.7% (148/252).
Number of outliers (bottom range): 104 - 41.3% (104/252).
Number of outliers: 5340
Number of outliers (top range): 5227 - 97.9% (5.227/5.340).
Number of outliers (bottom range): 113 - 2.1% (113/5.340).
Number of outliers: 7238
Number of outliers (top range): 7229 - 99.9% (7.229/7.238).
Number of outliers (bottom range): 9 - 0.1% (9/7.238).
delay)Basic assumption: Higher accuracy = lower delay
To evaluate it, we plot timestamps as follows (being “n” each row in the data file):
We calculate planning timestamp(n-1) - planning timestamp(n) to detect wrong planning timestamp values.
## Adding missing grouping variables: `scheduler`
## Adding missing grouping variables: `scheduler`
## Adding missing grouping variables: `scheduler`
## Adding missing grouping variables: `scheduler`
## Adding missing grouping variables: `scheduler`
## Adding missing grouping variables: `scheduler`
## Adding missing grouping variables: `scheduler`
## Adding missing grouping variables: `scheduler`
## Adding missing grouping variables: `scheduler`
## Adding missing grouping variables: `scheduler`
| # exp | Type scheduler | Device | # records | Start | End | Min | Bottom whisker | 1st Qu. | Median | Mean | 3rd Qu. | Top whisker | Max |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| #1 | Advanced | BQ Aquaris V | 21391 | 2020-03-30 12:15:03 | 2020-04-14 13:07:57 | 0 | 60.159 | 60.463 | 60.565 | 60.735 | 60.666 | 60.970 | 1323.965 |
| Nvidia Shield Tablet | 21453 | 2020-03-30 12:15:03 | 2020-04-14 13:08:14 | 0 | 58.629 | 60.012 | 60.179 | 60.560 | 60.980 | 62.415 | 191.099 | ||
| Xiaomi Mi A1 | 21465 | 2020-03-30 12:23:27 | 2020-04-14 13:07:50 | 0 | 58.936 | 60.039 | 60.224 | 60.501 | 60.796 | 61.887 | 628.508 | ||
| Basic | Honor 9 | 21574 | 2020-03-30 12:00:03 | 2020-04-14 13:01:35 | 0 | 59.907 | 60.014 | 60.025 | 60.243 | 60.103 | 60.235 | 296.462 | |
| Motorola Moto G | 21619 | 2020-03-30 12:00:07 | 2020-04-14 13:01:16 | 0 | 59.980 | 60.008 | 60.016 | 60.117 | 60.027 | 60.055 | 120.719 | ||
| #2 | Advanced | Honor 9 | 19377 | 2020-04-15 11:00:02 | 2020-04-29 12:59:15 | 0 | 59.426 | 60.063 | 60.109 | 62.794 | 60.738 | 61.748 | 44935.248 |
| Motorola Moto G | 19735 | 2020-04-16 12:09:53 | 2020-04-30 12:58:44 | 0 | 59.733 | 60.032 | 60.064 | 61.441 | 60.377 | 60.895 | 186.625 | ||
| Basic | BQ Aquaris V | 20146 | 2020-04-15 11:16:26 | 2020-04-29 13:00:20 | 0 | 60.045 | 60.247 | 60.262 | 60.351 | 60.385 | 60.592 | 94.369 | |
| Nvidia Shield Tablet | 3528 | 2020-04-15 11:16:18 | 2020-04-17 22:27:06 | 0 | 59.113 | 60.002 | 60.099 | 60.388 | 60.979 | 62.029 | 64.777 | ||
| Xiaomi Mi A1 | 10832 | 2020-04-15 11:16:23 | 2020-04-23 00:52:00 | 0 | 59.285 | 60.009 | 60.269 | 60.352 | 60.579 | 61.429 | 168.009 |
| Device | Type scheduler | # outliers | # outliers positive | # outliers negative |
|---|---|---|---|---|
| BQ Aquaris V | Advanced | 1698 | 888 | 810 |
| BQ Aquaris V | Basic | 3085 | 2517 | 568 |
| Honor 9 | Advanced | 455 | 263 | 192 |
| Honor 9 | Basic | 5066 | 5048 | 18 |
| Motorola Moto G | Advanced | 2589 | 2560 | 29 |
| Motorola Moto G | Basic | 3653 | 3287 | 366 |
| Nvidia Shield Tablet | Advanced | 197 | 181 | 16 |
| Nvidia Shield Tablet | Basic | 4 | 3 | 1 |
| Xiaomi Mi A1 | Advanced | 359 | 180 | 179 |
| Xiaomi Mi A1 | Basic | 21 | 19 | 2 |
## Adding missing grouping variables: `scheduler`
## Adding missing grouping variables: `scheduler`
## Adding missing grouping variables: `scheduler`
## Adding missing grouping variables: `scheduler`
## Adding missing grouping variables: `scheduler`
## Adding missing grouping variables: `scheduler`
## Adding missing grouping variables: `scheduler`
## Adding missing grouping variables: `scheduler`
## Adding missing grouping variables: `scheduler`
## Adding missing grouping variables: `scheduler`
| # exp | Type scheduler | Device | # records | Start | End | Min | Bottom whisker | 1st Qu. | Median | Mean | 3rd Qu. | Top whisker | Max |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| #1 | Advanced | BQ Aquaris V | 21391 | 2020-03-30 12:15:03 | 2020-04-14 13:07:57 | 45.580 | 60.157 | 60.462 | 60.565 | 60.739 | 60.666 | 60.971 | 1324.016 |
| Nvidia Shield Tablet | 21453 | 2020-03-30 12:15:03 | 2020-04-14 13:08:14 | -103.246 | 58.723 | 60.002 | 60.182 | 60.565 | 60.982 | 62.392 | 190.955 | ||
| Xiaomi Mi A1 | 21465 | 2020-03-30 12:23:27 | 2020-04-14 13:07:50 | -64.584 | 59.123 | 60.042 | 60.232 | 60.504 | 60.804 | 61.888 | 628.652 | ||
| Basic | Honor 9 | 21574 | 2020-03-30 12:00:03 | 2020-04-14 13:01:35 | 0.566 | 59.843 | 60.010 | 60.028 | 60.246 | 60.122 | 60.290 | 355.912 | |
| Motorola Moto G | 21619 | 2020-03-30 12:00:07 | 2020-04-14 13:01:16 | 0.089 | 59.744 | 59.976 | 60.023 | 60.120 | 60.131 | 60.363 | 120.747 | ||
| #2 | Advanced | Honor 9 | 19377 | 2020-04-15 11:00:02 | 2020-04-29 12:59:15 | -70.030 | 59.090 | 60.062 | 60.118 | 62.797 | 60.744 | 61.758 | 44935.732 |
| Motorola Moto G | 19735 | 2020-04-16 12:09:53 | 2020-04-30 12:58:44 | -101.278 | 59.101 | 60.024 | 60.080 | 61.444 | 60.648 | 61.573 | 225.801 | ||
| Basic | BQ Aquaris V | 20146 | 2020-04-15 11:16:26 | 2020-04-29 13:00:20 | 59.542 | 60.043 | 60.247 | 60.262 | 60.354 | 60.383 | 60.587 | 94.369 | |
| Nvidia Shield Tablet | 3528 | 2020-04-15 11:16:18 | 2020-04-17 22:27:06 | 59.113 | 59.113 | 59.997 | 60.099 | 60.405 | 60.979 | 62.062 | 64.704 | ||
| Xiaomi Mi A1 | 10832 | 2020-04-15 11:16:23 | 2020-04-23 00:52:00 | 26.323 | 59.305 | 60.021 | 60.269 | 60.358 | 60.578 | 61.402 | 168.004 |
BQ Aquaris V
How does delay behave in the middle 50% of data (IRQ)?
Advance scheduler shows worse delay times. In addition, the central cluster shows steadily increased values of delay over time. In contrast, the basic scheduler shows uniform delays over time adn concentrated aoru 0.3 sec. Teh basic scheduler’s boxplot shows a thinner box than the advance scheduler’s. There seems no clear relationship between battery levels and delays, which suggests that low battery levels do not impact (or at least not significally) on the planned execution time of a task.
What happens when battery is going to be fully recharged?
Honor 9
How does delay behave in the middle 50% of data (IRQ)?
## Warning: Removed 31 row(s) containing missing values (geom_path).
## Warning: Removed 22 row(s) containing missing values (geom_path).
Motorola Moto G
See the shiny app! (TO BE UPDATED)
delay over battery - without outliers)Higher reliability = higher percentage of alarms executed / expected alarms for the experiment time frame
Distribution of data based on a five number summary (“minimum”, first quartile (Q1), median, third quartile (Q3), and “maximum”).
Basic statistics applied to delay and device_name. Given a device, how do the box plots change on each scheduler?